Field sequence detector, method and video device

ABSTRACT

A field sequence detector determines the field sequence of a series of fields of video by assessing the vertical frequency content of hypothetical de-interlaced images. Hypothetical images are formed from a currently processed field and an adjacent (e.g. previous or next) field. If the vertical frequency content is relatively high (e.g. above ½ the Nyquist frequency for the image), the hypothetical image is assessed to be formed of improperly interlaced fields, belonging to different frames. If the frequency content is relatively low, the hypothetical image is assessed to be properly assembled from fields of the same frame. The field sequence in the series of fields may be detected from the assessed frequency content for several of said series of fields. Known field sequence, such as 3:2 pull-down, 2:2 pull down, and more generally m:n:l:k pull-down sequences.

FIELD OF THE INVENTION

The present invention relates generally to video devices and moreparticularly to a field sequence (e.g. 3:2/2:2 pull-down) detector.

BACKGROUND OF THE INVENTION

Moving pictures are stored as video. The moving pictures represent asequence of still images, that when displayed give the appearance ofmotion. The images may be stored as “frames” or “fields”. A framerepresents a single still image. By contrast, a field typicallyrepresents every second horizontal line (or row) in a still image. Moregenerally, a frame can be thought of as the combination of two fieldsthat form an image.

Fields are typically referred to as odd and even, with odd fieldcontaining odd lines of a frame, and even fields containing even linesof a frame.

Analog television in North America is broadcast in accordance with theNTSC standard, and uses interlaced fields. The NTSC standard calls forabout sixty (exactly 59.94) fields to be presented each second.Sequential fields are received and presented. The human eye perceivestwo sequential fields as a single frame.

Most cinema films are filmed at a rate of twenty-four images per second.Accordingly, cinema films have historically been converted for analogNTSC television using a process known as 3:2 pull-down. This conversionis more particularly illustrated in FIG. 1. As illustrated a sequence offilm images, represented as frames ABCDE . . . is divided into even andodd fields, and one field of every second frame is repeated. Theresulting field sequence isA_(O)A_(E)A_(O)B_(E)B_(O)C_(E)C_(O)C_(E)D_(O)D_(E)E_(O)E_(E)E_(O). Thus,fields from every second frame are presented in three fields, and fieldsformed from every second other frame are presented for only two fields.Twenty four frames are thus converted into sixty interlaced fields.

In Europe, analog television is transmit in accordance with the PAL orSECAM standards. These standards call for 50 interlaced fields persecond. Cinema films are converted for analog PAL/SECAM television usinga process known as 2:2 pull-down. This conversion is also illustrated inFIG. 1. As illustrated a sequence of film frames ABCDE . . . is used toform corresponding even and odd fields. Each even and odd field is shownonce every second frame. The resulting field sequence isA_(O)A_(E)B_(O)B_(E)C_(O)C_(E)D_(O)D_(E)E_(O)E_(E) . . . Twenty fourframes are thus converted into about fifty (i.e. forty eight) interlacedfields.

Newer television, computer and similar displays, however no longerdisplay interlaced video. Instead such displays display the videoprogressively, one line after the next. Accordingly, newer video outputdevices, such as for example digital versatile disk (DVD) players,computer games and the like, output video progressively, line by line.

Often, video to be presented by such progressive scan devices comes fromvideo provided as fields. For example, many DVDs still store video asfields of MPEG (or MPEG2) data, that must be de-interlaced. Moreover,many such sources store the video as fields in film mode (e.g. 3:2pull-down or 2:2 pull-down). Progressive scan devices must be able toaccurately assemble progressive scan frames from the interlaced data.They must therefore be able to detect the field sequence (often referredto as cadence) to correctly combine the fields to form frames. To thisend, many sources (such as DVDs) are coded with flags that are intendedto indicate whether stored video is stored in film mode or not.Unfortunately, video is often poorly edited: video is cut apart andreassembled in a way that destroys or alters the sequence. Similarly,video is often transferred or broadcast without these flags. In short,the flags cannot be relied upon.

As a result, other field sequence detectors (or cadence detectors) areknown. Some detectors compare the difference between pixels in adjacentfields. If the source video is stored in film mode, the differences willfollow a predictable pattern. Many of these detection circuits, however,cannot robustly distinguish between noise, slow movements, content thatoverlays interlaced and film mode material, high frequencies, andconventional interlaced content.

Other techniques include the analysis of motion vector data from fieldto field. Such techniques, however, are quite complex.

Accordingly, there remains a need for a field sequence detector andmethod that can easily and quickly detect the presence of film modevideo.

SUMMARY OF THE INVENTION

In accordance with an aspect of the present invention, a field sequencedetector determines the field sequence of multiple fields of video byassessing the vertical frequency content of hypothetical de-interlacedimages. Hypothetical images are formed from a currently processed fieldand an adjacent (e.g. previous or next) field. If the vertical frequencycontent in the hypothetical image is relatively high (e.g. above ½ theNyquist frequency for the image), the hypothetical image is assessed tocontain artifacts, and is likely formed of fields belonging to differentframes. If the frequency content is relatively low, the hypotheticalimage is assessed to be properly assembled from two complementary fieldsof the same frame. The field sequence in the series of fields may bedetermined using the assessed frequency content for several fields.

In accordance with an aspect of the present invention, there is provideda method of detecting a field sequence of a series of fields of video.The method includes: processing each of the series of fields byassessing the frequency content along the vertical direction of a firsthypothetical image formed by interleaving alternating rows from thecurrently processed field and rows from an adjacent field, to determinethe presence of high frequency vertical artifacts in the firsthypothetical image; and detecting the field sequence in the series offields, from the sequence of first hypothetical images determined tohave high frequency vertical artifacts.

In accordance with another aspect of the present invention, there isprovided a detector for detecting a field sequence of a series of fieldsof video. The detector includes: a first frequency analyzer forassessing vertical frequency content in first hypothetical images formedby interleaving alternating rows from a currently processed field androws from an adjacent field; a processor in communication with the firstfrequency analyzer to determine the sequence of the series of fields,from the assessed vertical frequency content for several currentlyprocessed fields of the series of fields.

In accordance with yet another aspect of the present invention, there isprovided a device for detecting a field sequence of a series of fieldsof video. The device includes: means for processing each of the seriesof fields to construct at least a portion of a first hypothetical imageformed by interleaving alternating rows from the currently processedfield and rows from an adjacent field; means for assessing the frequencycontent along the vertical direction of the first hypothetical image todetermine the presence of high frequency vertical artifacts in the firsthypothetical image; and means for detecting the field sequence in theseries of fields, by analyzing the assessed frequency content forseveral of the series of fields.

In accordance with yet another aspect of the present invention, a methodincludes, for each field of a plurality of video fields, constructing atleast a portion of an associated hypothetical image formed byinterleaving rows from the field and rows from an adjacent field in theplurality of video fields; assessing the frequency content along thevertical direction of each of the associated hypothetical images todetermine the presence of high frequency vertical artifacts therein;determining a field sequence in the plurality of fields from thesequence of those associated hypothetical images determined to have highfrequency vertical artifacts.

In accordance with yet another aspect of the present invention, there isprovided a method of processing a series of fields of video. For each ofthe fields, frequency content along the vertical direction of anassociated hypothetical image formed by interleaving alternating rowsfrom the field and rows from an adjacent field is assessed. Each of theassociated hypothetical images is classified as having or not havinghigh frequency vertical artifacts based on the assessing.

Aspects of the invention may be embodied as computer readableinstructions stored on a computer readable medium.

Other aspects and features of the present invention will become apparentto those of ordinary skill in the art upon review of the followingdescription of specific embodiments of the invention in conjunction withthe accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

In the figures which illustrate by way of example only, embodiments ofthe present invention,

FIG. 1 schematically illustrates the encoding of film frames, to asequence of interlaced film mode (3:2 pull-down and 2:2 pull-down)fields;

FIG. 2 is a simplified schematic block diagram of a video deviceincluding a field sequence detector, exemplary of an embodiment of thepresent invention;

FIG. 3 is simplified schematic block diagram of the field sequencedetector of FIG. 2;

FIGS. 4A and 4B are spectra of images excluding and including verticalartifacts, respectively;

FIGS. 5A to 5D are histograms illustrating the distributions of pixelsproximate high frequency vertical content for various hypotheticalimages;

FIG. 6 is a flow chart illustrating steps performed by the fieldsequence detector of FIG. 3, exemplary of embodiments of the presentinvention; and

FIGS. 7A and 7B illustrates weave patterns for de-interlacing exemplaryfields in 3:2 pull-down and 2:2 pull-down field sequences.

DETAILED DESCRIPTION

FIG. 2 illustrates a video device 10 for generating a progressive scanvideo output signal. Video device 10 includes a field sequence detector20, exemplary of embodiments of the present invention. Video device 10may be any device suited to decode and/or present digital video. Device10 may, for example, be a computing device, a television, a set-top boxfor receiving cable, satellite, broadcast or similar video signals, aDVD player, or the like.

As will become apparent, field sequence detector 20 determines ifadjacent fields of a series of fields likely originate with the same ordifferent frames. Sequence detector 20 relies on the observation thatcombining fields of different frames typically results in images havinghigh frequency artifacts in the vertical direction, while combiningcomplementary fields from the same frame does not result in suchartifacts.

As illustrated, video device 10 includes a video source 12 of digitizedvideo providing a sequence of video fields to a frame buffer 18. In theembodiment of FIG. 2 video source 12 provides decoded video stored, forexample, as MPEG1 or 2 video originating with MPEG source 14. MPEGsource 14 may, for example, be a source of MPEG encoded video, and mayfor example be a DVD, read by an appropriate reader. MPEG source 14could similarly be a demodulated MPEG stream carried by a satellite orcable television signal, or digital broadcast television signal. Decoder16 decodes the digital stream into rasterized video frames or fields.For example, for interlaced sources, decoder 16 decodes interlaced videointo odd and even fields and provides these fields to frame buffer 18.

Of course, this invention may be equally applicable to video that is notfrom an MPEG other than a DVD. For example, video could be formatted asMPEG4; H.264; Quicktime. Video source 12 could provide a digitizedversion of an analog video source such as a video cassette recorder. Anyaudio associated with video source 12 may be separately extracted andamplified in a conventional manner, not detailed herein.

In any event, the output of video source 12 is provided as fields ofrasterized video to frame buffer 18. Field sequence detector 20 isinterconnected with frame buffer 18 and video block 22.

As illustrated, video block 22 includes a de-interlacer 24, a linebuffer 26, an overlay generator 27, and a digital to analog converter(DAC) 28. Video block 22 converts pixel data in frame buffer 18, line byline into an analog output for display on a display such as a computermonitor, television, or the like. Data in the line to be converted bythe DAC is stored within line buffer, where it may be scaled oroverlayed with additional video information, in conventional waysunderstood by those of ordinary skill. Data provided to the line buffer26 is provided from frame buffer 18 by de-interlacer 24. Additionally,overlays in the form of graphics, subtitles or the like may be generatedby overlay generator 27.

De-interlacer 24 governs the order in which lines of video within framebuffer 18 are converted to analog video. The mode of operation ofde-interlacer 24 is controlled by field sequence detector 20. In analternate embodiment, video block 22 may output digital data directlywithout a DAC to drive a pixel-based digital display (such as an LCD,digital light processing (DLP) display, or the like).

As noted, odd fields contain odd lines of the video frame, while evenfields contain even lines of the video frame. Frame buffer 18 stores atleast three fields at any given time: the fields may be consideredprevious, current and next fields. Preferably, frame buffer 18 operatesfirst-in, first out: each time a new field arrives from video source 12,an old one is discarded, while previous, current and next fields arestored at the same locations, field after field. As will becomeapparent, de-interlacer 24 assembles frames of progressive video to beprovided to DAC 28 from current and next fields, or current and previousfields.

Decoder 16, frame buffer 18, display block 22 and film sequence detector20 may each be formed using conventional semiconductor design andproduction techniques appreciated by those of ordinary skill.

FIG. 3 further illustrates field sequence detector 20, exemplary ofembodiments of the present invention (and frame buffer 18).

As illustrated, field sequence detector 20 includes an addressgenerator/data decoder 30, for generating pixel values at addresseswithin frame buffer 18 to be processed as described below. The pixelvalues typically represent the color and/or intensity of the pixels.

Field sequence detector 20 relies on the frequency content in thevertical direction of hypothetical images formed by interleavingalternating lines from adjacent fields—the current and previous field,and/or hypothetical images formed by images alternating lines currentand next field video. If hypothetical images contain high frequencyartifacts, the corresponding source fields likely originate withdifferent frames, and should not be combined to form a de-interlacedimage.

In one embodiment, the frequency content in a vertical direction withinthe hypothetical images is assessed. Specifically, the frequency contentin a vertical direction around all or numerous pixels in thehypothetical images are determined, to classify the hypothetical imageas having relatively high (H) or low (L) vertical frequency.

To this end, in a first embodiment, address generator/data decoder 30sequentially generates the values within a column about a pixel ofinterest, in a hypothetical image formed by combining current andadjacent fields. That is, for each pixel having co-ordinates (x,y)within a hypothetical image, address generator/data decoder 30 generatesten addresses and corresponding pixel values from current, next andprevious fields: the address within buffer 18 of the pixel P_(A)(x,y)within the immediately previous field (corresponding to the pixel F(x,y)in the hypothetical image/frame) and the pixel value P(x,y); addressN_(A)(x,y) and pixel value N(x,y) in the immediately subsequent (i.e.next) field, as well as the addresses of the pixels directly above andbelow the pixel in the current field, the previous field, and the nextfield.

The addresses (denoted by subscript A) of the pixels may be representedmathematically as, P_(A)(x, y − 2) = F_(CPA)(x, y − 2)P_(A)(x, y) = F_(CPA)(x, y) P_(A)(x, y + 2) = F_(CPA)(x, y + 2)N_(A)(x, y − 2) = F_(CNA)(x, y − 2) N_(A)(x, y) = F_(CNA)(x, y)N_(A)(x, y + 2) = F_(CNA)(x, y + 2)C_(A)(x, y − 3) = F_(CNA)(x, y − 3) = F_(CPA)(x, y − 3)C_(A)(x, y − 1) = F_(CNA)(x, y − 1) = F_(CPA)(x, y − 1)C_(A)(x, y + 1) = F_(CNA)(x, y + 1); = F_(CPA)(x, y + 1)  andC_(A)(x, y + 3) = F_(CNA)(x, y + 3) = F_(CPA)(x, y + 3)where (x,y) represent coordinates within a hypothetical image/frame; P,N and C represent the previous, next and current fields; F_(CN)represents a hypothetical image formed by combining the current fieldwith the next field; and F_(CP) the current field with the previousfield. Notably, for convenience, addresses are expressed with referenceto a constructed image/frame. Co-ordinates could similarly be expressedas coordinates within the fields P, N and C.

Address generator/data decoder 30 further outputs the values of buffer18 at the generated addresses. To do so, address generator/data decoder30 may further include a buffer (not shown) to store data within framebuffer 18, at the identified pixel addresses. Pixel values at the tengenerated addresses may be read by address generator/data decoder 30from frame buffer 18 sequentially.

Now, using pixels within buffer 18 at the identified addresses, it ispossible to reconstruct a portion of a column in hypothetical imageF_(CN) or F_(CP), representative of a possible de-interlaced frame. Forexample, the pixel sequence

(x,y−3); P(x,y−2); C(x,y−1); P(x,y); C(x,y+1); P(x,y+2); C(x,y+3)

represents a column of seven pixels, within a hypothetical image F_(CP)constructed from the previous and current field, centered about thepixel at co-ordinate (x,y).

Similarly, the pixel sequence

C(x,y−3); N(x,y−2); C(x,y−1); N(x,y); C(x,y+1); N(x,y+2); C(x,y+3),

represents a portion of a column, within a hypothetical image F_(CN)constructed from current and next field, centered about the pixel atco-ordinate (x,y).

Field sequence detector 20 further includes two frequency analyzers:current and previous field (CP) frequency analyzer 44; and current andnext field (CN) frequency analyzer 38. Each frequency analyzer 38, 44performs a frequency analysis on a provided sequence of pixel values. Inthe depicted embodiment, frequency analysis is performed by eachanalyzer 38, 44 using seven vertically aligned pixels within an image.Of course, more or fewer pixels could be used for frequency analysis.

Specifically, frequency analyzer 38 performs a frequency analysis on thepixel sequence,

C(x,y−3); P(x,y−2); C(x,y−1); P(x,y); C(x,y+1); P(x,y+2); C(x,y+3)

at addresses C_(A)(x,y−3); P_(A)(x,y−2); C_(A)(x,y−1); P_(A)(x,y);C_(A)(x,y+1); P_(A)(x,y+2); C_(A)(x,y+3) within buffer 18.

Analyzer 44 performs frequency analysis on the pixel sequence C(x,y−3);N(x,y−2); C(x,y−1); N(x,y); C(x,y+1); N(x,y+2); C(x,y+3) at addressesC_(A)(x,y−3); N_(A)(x,y−2); C_(A)(x,y−1); N_(A)(x,y); C_(A)(x,y+1);N_(A)(x,y+2); C_(A)(x,y+3), within buffer 18.

As will become apparent, each frequency analyzer 38, 44 may perform afast Fourier, discrete Fourier, wavelet, discrete wavelet or similarfrequency analysis of the values of the provided pixel sequence.

Notably, an image formed by interlacing two fields originating with twodifferent frames typically results in visible artifacts often describedas a “combing pattern”, “mice teeth”, “tearing”, “feathering”, or“serrations”. The frequency content of a sequence of vertical pixelshaving such artifacts is generally quite high, and typically exhibitsdistinct characteristics. Conversely, the frequency content in a columnof pixels of an image formed of even and odd fields of the same frame isrelatively low. Specifically, FIGS. 4A and 4B show the verticalfrequency content of representative images without and with verticalartifacts, respectively. Notably, the frequency content at frequenciesgreater than ¼ the sampling frequency (i.e. Nyquist/2) is morepronounced for images including the vertical artifacts (FIG. 4B), thanfor images that do not include such artifacts (FIG. 4A).

Detector 20 takes advantage of the frequency content of hypotheticalimages formed of adjacent fields, to detect field sequences, such as 3:2pull-down; 2:2 pull-down or other field sequences.

Now, each analyzer 38, 44 outputs a metric of the spectral quality ofthe analyzed pixels. In the depicted embodiment, the metric calculatedby each analyzer 38, 44 is the amplitude of the dominant frequencycomponent having a frequency greater some threshold frequency (typically¼ the sampling frequency or the Nyquist frequency/2). Analyzers 38, 44may do this by locating the local maximum of the portion of the spectrumabove this threshold frequency, and outputting the associated amplitude.

Additionally, analyzers 38, 44 may further optionally verify that thislocal maximum does likely represent the dominant frequency, resultingfrom vertical artifacts. To do so, analyzers 38, 44 may verify that thevertical frequency content proximate the pixel of interest (x,y),approximates the vertical frequency characteristics of a hypotheticalimage including artifacts. Specifically, as illustrated in FIG. 4B, thefrequency content attributable to vertical artifacts, only manifestitself at frequencies in the upper half of the frequency spectrum.Further, the amplitude of the associated frequencies is generallyincreasing. This spectral pattern has been found to be a fingerprint ofthe “weaving” artifact, described. Thus, analyzers 38, 44 mayadditionally verify that this, or a similar, fingerprint is present foreach pixel of interest. Analyzers 38, 44 may accomplish this bydetermining that the amplitude of the spectral content is generallyincreasing between 0.5 and 0.75 of the Nyquist frequency. Alternatively,analyzers 38, 44 may determine that the local maxima in the spectralregions of 0.5-0.625, 0.625-0.75 and 0.75-875 of the Nyquist frequencyare increasing. Other techniques are possible. In any event, ifanalyzers 38, 44 fail to verify the presence of the fingerprint, theymay output a value of “0”, otherwise the maxima of the spectral contentin the top half of the spectrum may be output.

An additional difference calculator 52, calculates the absolutedifference between the pixel at co-ordinate (x,y) in the next image N,and previous image, P.

Processor 50 is in communication with analyzers 38 and 44, anddifference calculator 52 by way of filter 46 and accumulator banks 42,48, and 50 to read outputs of analyzers 38 and 44, and differencecalculator 52 to assess the frequency content of each hypothetical imageand detect a field sequence. Broadly, processor 50 assesses thefrequency content to classify the hypothetical images as containing highfrequency content (H) expected as a result of artifacts, or lowerfrequency content, expected in properly formed images.

Filter 46 may remove or smooth data that is not in-line with previouslygenerated data by averaging or otherwise filtering frequency metrics formultiple adjacent pixels.

For each current field, the output of filter 46 is provided toaccumulator banks 42, 48. Accumulator banks form a cumulative histogramreflecting the outputs of frequency analyzers 38, 44 for eachhypothetical image for an entire current field. As will become apparent,accumulator banks 42 and 48 count the number of analyzed pixels forwhich frequency metrics fall within defined ranges, and thus provide ahistogram of the cumulative frequency metric for the hypothetical image.As such, each accumulator bank 42, 48 may include multiple registers,one corresponding to each defined range. Specifically, accumulator banks42 and 48 count the number of pixels for which the output metric offilter 38, 44 (i.e. amplitude of the dominant frequency above ¼ thesampling frequency) is within one of several ranges. The ranges may beempirically determined. Alternatively, the ranges may be dynamicallyupdated by processor 50.

The output of difference calculator 52 is similarly output toaccumulator bank 54. Accumulator bank 54 counts the number of pixels forwhich abs (N(x,y)−P(x,y)) is below a threshold.

Operation of processor 50 in manners exemplary of embodiments of thepresent invention is controlled through a series of processor executableinstructions that may be loaded into instruction RAM/ROM or firmware(not specifically illustrated) associated with processor 50. The RAM/ROMmay be external to processor 50 or integral therewith. Processorreadable instructions may be loaded from a computer readable medium (notshown).

In the depicted embodiment, fields are processed sequentially. However,a person will realize that multiple fields could be buffered andprocessed. As illustrated, for each currently processed field (i.e. theN^(th) field within a sequence) provided by video source 12, addressgenerator/data decoder 30 generates sets of ten addresses for pixelswithin the current field. Sets of ten pixel addresses spanning at leasta portion of each field are generated sequentially. Addressgenerator/data decoder 30 accordingly generates sets of ten addressesfor each (x,y) coordinate provided to address generator/data decoder 30.

For each current field, addresses for multiple (x,y) co-ordinates,between (x_(min), y_(min)) and (x_(max), y_(max)) are generated. Valuesof x_(min), x_(max), y_(min) and y_(max) may be stored in registers 32,34, 36 and 40, respectively. The values of (x_(min), y_(min)) and(x_(max), y_(max)) are typically fixed, and may be chosen to allowanalysis of pixels within a representative region of each hypotheticalimage. In one embodiment (x_(min), y_(min)) and (x_(max), y_(max)) arechosen so that all pixels forming the image are analyzed. Of course,pixels within smaller representative regions could be analyzed.Similarly, non-continuous samples of the images could be used.

In the depicted embodiment, address generator/data decoder 30 generates,sets of data at addresses C_(A)(x,y−3); P_(A)(x,y−2); C_(A)(x,y−1);P_(A)(x,y); C_(A)(x,y+1); P_(A)(x,y+2); C_(A)(x,y+3) and C_(A)(x,y−3);N_(A)(x,y−2); C_(A)(x,y−1); N_(A)(x,y); C_(A)(x,y+1); N_(A)(x,y+2);C_(A)(x,y+3), for x_(min)≦x≦x_(max), and y_(min)≦y≦y_(max).

Frequency analyzer 44 uses the values of pixels at the generatedaddresses to determine frequency metrics, representative of the verticalfrequency content at the various pixels, of a hypothetical image formedof the current (N^(th)) field, C, and previous ((N−1)^(th)) field P.

Frequency analyzer 38 similarly determines metrics, representative ofthe vertical frequency content at the various pixels, of a hypotheticalimage formed of the current (N^(th)) field C, and next ((N+1)^(th))field N.

Difference calculator 52 calculates abs (N(x,y)−P(x,y)).

After the metrics for one pixel in each of the current-next field, andcurrent-previous field hypothetical images have been determined byfrequency analyzers 38 and 44, address generator/data decoder 30, mayadvance the row and then column address within the fields, for eachhypothetical image, so that the vertical frequency about multiple pixelswithin the hypothetical image may be assessed. Filter 46 may filterfrequency metrics associated with multiple adjacent pixels, to provide afrequency metric for each pixel in the hypothetical image.

Accumulator banks 42, 48 and 54 are then updated to count the number ofpixels having frequency metrics in defined ranges. Possible histogramsfor hypothetical images using four possible ranges (RANGE1, RANGE2,RANGE3, and RANGE4) are depicted in FIGS. 5A to 5D. As illustrated,images formed of complementary fields of the same frame should typicallyexhibit histograms as depicted in FIG. 5A or 5B, and may be assessed tohave relatively low (L) vertical frequency content. Images exhibitingweave artifacts should exhibit histograms as depicted in FIG. 5C or 5D,and may be assessed to have relatively high (H) vertical frequencycontent.

Processor 50, in turn, is provided with metrics of the frequency contentin the vertical direction of the two hypothetical images formed from thecurrent and previous fields and the current and next field, for eachcurrent field, as stored in accumulator banks 42, 48. Processor 50 nexttries to determine whether current and next or current and previousfields are best combined to form a de-interlaced image. To do so,processor 50 determines whether the histogram stored in accumulatorbanks 42 and 48 more closely resembles the histogram in FIG. 5A or 5B,or FIG. 5C or 5D, and the hypothetical images should be assessed to haverelatively low (L) or high (H) vertical frequency content.

Optionally, the output of accumulator 54, storing the number of pixelsin previous and current fields for which the difference is less than thethreshold, may be used by processor 50 to better determine whether ahistogram reflects high frequency artifacts, or merely reflects highfrequencies present in the image. Specifically, if the difference valueis zero (or near zero) for all (or nearly all) pixels in the addressedregion, current-next field frequency assessment should be the same ascurrent-previous field frequency assessment as determined for theimmediately preceding processed field. Vertical high frequency contentin the current-next field can then be assumed to be the result highfrequency content in the original image, and not artifacts.

Notably, the contents of accumulator banks 42, 48 need not be exact, butrather only relative. That is, the contents of accumulator banks 42, 48are only used to assess whether the frequency content in a verticaldirection of hypothetical images is higher (denoted herein as H) thanthe anticipated frequency content (denoted herein as L) for properlyde-interlaced images. Accordingly, any suitable technique may be used toassess whether a histogram for a particular hypothetical image moreclosely resembles FIGS. 5A, 5B or FIGS. 5C, 5D, and classify the image.This may done by as simply as determining the fraction of pixels aboutwhich the dominant frequency in the vertical direction is in the highestamplitude segment, as counted by accumulator banks 42, 48 exceeding somethreshold. In this case the hypothetical image may be assessed to havehigher frequency content in a vertical direction (H) than theanticipated frequency content for a properly de-interlaced image.

Conveniently, values in accumulator banks 42, 48 may be furtherprocessed using past values of accumulator banks 42, 48. Specifically,once a hypothetical image has been determined to not contain artifacts,the corresponding histogram may be considered a template histogram, andmay be used as a comparator or filter for future histograms. Histogramsin accumulator banks 42, 48 for future hypothetical images, may befiltered by subtracting or otherwise applying the template histogram.The modified results better reflect content attributable to artifacts ornoise.

Once frequency content in the vertical direction for hypothetical imagesfor multiple current fields has been assessed, processor 50 may estimatethe field sequence of the source, field by field. Specifically, forevery current field, the hypothetical current-previous field image andcurrent-next field image frequency assessments are compared to knownpatterns to determine the field sequence of the decoded image.

For example, consider adjacent fields in a 3:2 pull-down sequence

A_(O)A_(E)A_(O)B_(E)B_(O)C_(E)C_(O)C_(E)D_(O)D_(E)E_(O)E_(E)E_(O)F_(E)F_(O). . .

The vertical frequency content of hypothetical images formed fromcurrent and previous fields will typically be

HLLHLHLLHLHLLHL . . .

Similarly, the vertical frequency content of hypothetical images formedfrom current and next fields will be

LLHLHLLHLHLLHL . . .

For adjacent fields in a 2:2 pull-down sequence

A_(O)A_(E)B_(O)B_(E)C_(O)C_(E)D_(O)D_(E)E_(O)E_(E) F_(O)F_(E) . . .

the vertical frequency content in hypothetical images formed fromcurrent and previous fields will typically be

HLHLHLHLHLHL . . .

Likewise, the vertical frequency content of hypothetical images formedfrom current and next fields will be

LHLHLHLHLHL . . .

For adjacent fields in a 5:5 pull down sequence

A_(O)A_(E)A_(O)A_(E)A_(O)B_(E)B_(O)B_(E)B_(O)B_(E)C_(O)C_(E)C_(O)C_(E)C_(O)D_(E)D_(O)D_(E)D_(O)D_(E). . .

the vertical frequency content in hypothetical images formed of currentand previous fields will be

HLLLLHLLLLHLLLLHLLL . . .

and of images formed of current and next (CN) fields

LLLLHLLLLHLLLLHLLLLHLLLL . . .

Similarly, for adjacent fields in a 8:7 pull down sequenceA_(O)A_(E)A_(O)A_(E)A_(O)A_(E)A_(O)A_(E)B_(O)B_(E)B_(O)B_(E)B_(O)B_(E)B_(O)C_(E)C_(O)C_(E)C_(O)C_(E)C_(O)C_(E)C_(O). . .

the vertical frequency content in hypothetical images formed of currentand previous fields will typically be

HLLLLLLLHLLLLLLHLLLLLLLH . . .

and of images formed of current and next (CN) fields

LLLLLLLHLLLLLLHLLLLLLL . . .

More generally, for m:n pull-down sequence, the vertical frequencycontent in hypothetical images formed of current and next fields formultiple fields will be

(L)_(m-1)H(L)_(n-1)H(L)_(m-1)H(L)_(n-1) . . .

and the vertical frequency content in hypothetical images formed ofcurrent and previous fields will be

H(L)_(m-1)H(L)_(n-1)H(L)_(m-1)H(L)_(n-1) . . .

Similarly, for a m:n:l:k pull-down sequence (like, for example, 2:4:4:2,2:3:3:2, 2:3:2:3:2:2 sequences), the vertical frequency content inhypothetical images formed of current and next fields will be

(L)_(m-1)H(L)_(n-1)H(L)_(l-1)H(L)_(k-1)H(L)_(m-1) . . .

and the vertical frequency content in hypothetical images formed ofcurrent and previous fields will be

H(L)_(m-1)H(L)_(n-1)H(L)_(l-1)H(L)_(k-1)H(L)_(m-1) . . .

Using vertical frequency values for current and next field and currentand previous field hypothetical images, processor 50 detects threepossible modes:

mode 0=pull-down mode not detected; switch to de-interlacing;

mode 1=weave current field to next field to form frame; and

mode 2=weave current field to previous field to form frame

Steps S600 performed by processor 50 to detect a field sequence aredetailed in FIG. 6. Specifically, processor 50 may initially detect themode by determining whether the current-next or current-previous fieldcontains artifacts, by analysing the content of accumulator bank 42 (or48) in steps S606-S622. As described, the histogram stored in one orboth of accumulator bank 42 (or 48) may be compared to known histograms,to determine whether current and next, or current and previous fieldsmay be combined, and thus whether mode 1 or 2 should be used. If thecurrent and next field accumulator 42 indicates low frequency verticalcontent (L), mode 1 may be selected (S608-S612). If the current andprevious field accumulator 48 indicates low frequency vertical content(L), mode 2 may be selected (S608-S612). Optionally, the values inaccumulator bank 42 (or 48) may be adjusted prior to comparison usingtemplate histogram values for a properly de-interlaced image, asdescribed above, if available in step S602 and S604.

Additionally, an indicator of the vertical frequency content formultiple images may be buffered. After the indicator (H or L) formultiple fields has been buffered, 3:2, 2:2 or other pull-down sequencemay be detected in step S624. Specifically, if high frequency verticalcontent is assessed in hypothetical images for two non-adjacent fieldsin every five fields (i.e. the pattern . . . HLLHLHLLHLHLLH . . . ) a3:2 sequence may be detected. Likewise if the pattern . . . HLHLHLHLHLH. . . , a 2:2 pull-down sequence may be detected.

Once a known field sequence has been determined, consistency of the modedetermined in step S624 and S626 may be verified in step S608- and S614using the other accumulator banks 48 (or 42) for each field. That is, anassessment is made to ensure the complementary pattern ( . . .LLHLHLLHLHLL . . . for 3:2 pull-down; . . . LHLHLHLHL . . . for 2:2pull-down) has been generated by the other analyzer 44, 38. If not, thecurrent pull-down sequence may be re-verified in steps S624-S628. Theselected mode for the field may still be based on which of banks 42 and48 indicate low frequency content, or optionally mode 0 may be assumed.

Steps S624, S626 and S608, S614 may similarly detect and verify avariety of other field sequences, including but not limited to 5:5pull-down; 8:7 pull-down; edited 3:2 pull-down; and the like.

Once a known pull-down sequence is recognized and verified, processor 50may update the template histogram used in step S604 in step S628, usingthe contents of accumulator banks 42 or 48 for a hypothetical imagewithout artifacts.

Conveniently, the output of frequency analyzers 38 and 44 andaccumulator banks 42 and 48 may be quite sensitive to poor editing,(e.g. added commercials, cut scenes due to film ratings, etc), stillimages with changing subtitles, wrongly inserted scene changes,artificial field/frame drops to reduce running time, or other anomaliesto the film mode sequence.

In any event, after mode detection, an indicator of the detected mode isprovided by detector 20 to de-interlacer 24. De-interlacer 24accordingly assembles frames for display from odd and even fields inbuffer 18 for presentation on a progressive scan video display. Asnoted, in mode 1 de-interlacer 24 weaves the current field with the nextfield in buffer 18 to form the progressive frame. In mode 2,de-interlacer 24 current field with the previous field to form theframe. In mode 0, video is de-interlaced in a conventional manner. Forexample, adjacent fields may be weaved and/or interpolated (bobbed)together, using conventional de-interlacing techniques understood bythose of ordinary skill. Exampling de-interlacing modes for 3:2 and 2:2pull-down sequences are depicted in FIGS. 7A and 7B, respectively.

Even more generally, detector 20 may be used to detect frame repetitionsequences in a sequence of non-interlaced frames. Such a framerepetition sequence may, for example, manifest itself in progressivescan frames formed from 3:2 pull-down or 2:2 pull-down sources.Similarly, frames may be repeated in animated or computer generatedframes. Knowledge of the frame sequence may be used to more effectivelyconvert between frame rates, by taking into account the repetition offrames in the video source.

In order to use detector 20 to detect frame sequences, detector 20 isprovided with fields that would be formed from adjacent frames. Forexample, even lines of current frame, odd lines of the previous and nextframes may be stored in, or extracted from, frame buffer 18. Detector 20would continue to operate as described above. Analyzers 38 and 44 wouldthus assess frequency content in hypothetical images formed fromprevious-current, and current-next frames. Low frequency content wouldbe indicative of hypothetical images formed from the same frame, andtherefore frame repetition, while high frequency content would beindicative of hypothetical images formed from different frames. Postfrequency-content assessment processing could then determine framerepetition, for frame rate conversion and optional interpolation.

Although the above depicted embodiment generally relies on the use oftwo frequency analyzers 38 and 44 and accumulator banks 42 and 48, itshould now be apparent that a variant of the detector could be formedusing only a single analyzer 38 or 44 and associated accumulator bank 42or 48. Only current-previous field hypothetical images or current-nexthypothetical images would be analyzed. Verifications in steps S606-S608would thus be eliminated. Such a detector would likely function asdescribed, but would likely not be as robust, and may not work for somepeculiar field sequences.

Additionally, although the described embodiments rely on the use ofhistograms to more accurately determine the vertical frequency contentof each hypothetical image, a person of ordinary skill will appreciatethat use of such histograms is not necessary and may be varied. Forexample, if each frequency analyzer 38, 44 can accurately assess whetherthe vertical spectral content for a particular pixel does or does notrepresent the presence of a “weaving” artifact, analyzers 38, 44 couldsimply output a binary value for each pixel, reflective of whether ornot the artifact was present near the pixel. By simply counting thenumber of pixels reflective of the artifact, processor 50 may assesswhether the artifact is likely present image. Of course, as the outputof analyzers 38, 44 becomes more binary, analyzers 38, 44 should be moreprecisely able to determine frequency content. The use of histograms,thus allows for more qualitative, less exact, outputs from analyzers 38,44. In a further alternate embodiment, the raw output of frequencyanalysers 38, 44 could be stored for each pixel for further processingby processor 50. In this way, a three dimensional frequency profile ofeach hypothetical image would be generated, for analysis by processor50. Using this profile, processor 50 may assess the frequency content ofhypothetical images.

Although the above embodiments have been depicted in the context of aprogrammable processor 50 and associated hardware frequency analyzers 38and 44 and accumulator banks 42 and 48, a person of ordinary skill willreadily appreciate that methods exemplary of embodiments of theinvention could readily be embodied entirely in software, and stored ona computer readable medium. Similarly, processor 50 and associatedsoftware could be replaced partially or entirely with hardware.

Of course, the above described embodiments are intended to beillustrative only and in no way limiting. The described embodiments ofcarrying out the invention are susceptible to many modifications ofform, arrangement of parts, details and order of operation.

The invention, rather, is intended to encompass all such modificationwithin its scope, as defined by the claims.

1. A method of detecting a field sequence of a series of fields ofvideo, said method comprising: processing each of said series of fieldsby assessing the frequency content along the vertical direction of afirst hypothetical image formed by interleaving alternating rows fromthe currently processed field and rows from an adjacent field, todetermine the presence of high frequency vertical artifacts in saidfirst hypothetical image; detecting said field sequence in said seriesof fields, from the sequence of first hypothetical images determined tohave high frequency vertical artifacts.
 2. The method of claim 1,wherein said assessing comprises performing a frequency analysis ofpixels within a column about addressed pixels of said first hypotheticalimage.
 3. The method of claim 2, wherein said assessing comprisesperforming a wavelet analysis of said pixels within said column aboutsaid addressed pixels.
 4. The method of claim 2, wherein said assessingcomprises performing a Fourier analysis of said pixels within saidcolumn about said addressed pixels.
 5. The method of claim 1, whereinsaid detecting comprises recognizing high frequency content along thevertical direction of said first hypothetical image, for a definedpattern of said series of fields.
 6. The method of claim 1, wherein saidassessing comprises using multiple frequency analyses along columns ofsaid first hypothetical image to classify said first hypothetical imageas having or not having high frequency content in a vertical direction.7. The method of claim 1, wherein said assessing comprises determiningmagnitudes of dominant frequency components in the vertical direction,above a threshold frequency, proximate addressed pixels in said firsthypothetical image.
 8. The method of claim 7, wherein said assessingcomprises analysing a distribution of said magnitudes and classifyingsaid first hypothetical image as having high frequency content in thevertical direction, based on said distribution.
 9. The method of claim8, further comprising filtering said distribution of magnitudes, using adistribution of magnitudes of dominant frequencies, above saidthreshold, in a hypothetical image not having vertical artifacts. 10.The method of claim 1, wherein said field sequence is detected as a 3:2pull down sequence, in response to assessing two non-sequential ones ofsaid first hypothetical images for every five fields as having highfrequency content in the vertical direction.
 11. The method of claim 1,wherein said field sequence is detected as a 2:2 pull down sequence insaid series of fields, in response to assessing said first hypotheticalimage as having high vertical frequency content for every alternatingone of said series of fields.
 12. The method of claim 1, wherein saidfirst hypothetical image is formed by interleaving alternating rows fromthe currently processed field and rows from an immediately next field,said method further comprising: sequentially processing each of saidseries of fields by constructing at least a portion of a secondhypothetical image formed by interleaving alternating rows from thecurrently processed field and rows from an immediately previous field;assessing the frequency content along the vertical direction of saidsecond hypothetical image to determine the presence of a high frequencyvertical artifacts in said second hypothetical image.
 13. The method ofclaim 12, wherein said currently processed field and immediately nextfield are de-interlaced in response to detecting relatively lowfrequency content along the vertical direction of said firsthypothetical image.
 14. The method of claim 12, wherein said currentlyprocessed field and immediately previous field are de-interlaced inresponse to detecting relatively low frequency content along thevertical direction of said second hypothetical image.
 15. The method ofclaim 13, further comprising forming differences between pixels in saidprevious field and pixels in said next field, and wherein said assessingcomprises using said differences, for each of said currently processedfields to classify each of said first and second hypothetical images.16. The method of claim 1, wherein said field sequence is detected as am:n:l:k pull-down sequence in response to detecting a pattern ofrelatively high (H) and relatively low (L) frequency content in thevertical direction, for m+n+l+k sequential ones of said firsthypothetical images, said pattern corresponding to(L)_(m-1)H(L)_(n-1)H(L)_(l-1)H(L)_(k-1)H(L)_(m-1) . . .
 17. A detectorfor detecting a field sequence of a series of fields of video, saiddetector comprising: a first frequency analyzer for assessing verticalfrequency content in first hypothetical images formed by interleavingalternating rows from a currently processed field and rows from anadjacent field; a processor in communication with said first frequencyanalyzer to determine said sequence of said series of fields, from saidassessed vertical frequency content for several currently processedfields of said series of fields.
 18. The detector of claim 17, whereinsaid first frequency analyzer determines vertical frequency contentproximate individual pixels in said first hypothetical images.
 19. Thedetector of claim 18, further comprising an address decoder for decodingaddresses of pixels in said currently processed field and said adjacentfield to assess said vertical frequency content proximate saidindividual pixels.
 20. The detector of claim 19, wherein said addressdecoder decodes addresses for selected ones of said individual pixels ina programmed range.
 21. The detector of claim 18, wherein said firstfrequency analyzer is operable to determine a magnitude of a dominantfrequency, above a threshold frequency, in said vertical frequencycontent proximate each ones of said individual pixels in said firsthypothetical images.
 22. The detector of claim 21, further comprising anaccumulator bank to store a distribution of said magnitudes of saiddominant frequency for each of said first hypothetical images, saidaccumulator bank in communication with said processor, and wherein saidprocessor classifies each hypothetical image as having high frequencycontent in a vertical direction, based on said accumulator bank.
 23. Thedetector of claim 22, wherein said processor is further operable toadjust a content of said accumulator bank, to filter said content ofsaid accumulator bank based on a distribution of said magnitudes for ahypothetical image without artifacts.
 24. The detector of claim 17,wherein said first frequency analyzer analyzes hypothetical imagesformed by interleaving alternating rows from the currently processedfield and rows from an immediately next field, said detector furthercomprising: a second frequency analyzer for assessing vertical frequencycontent in second hypothetical images formed by interleaving alternatingrows from the currently processed field and rows from a previous field.25. The detector of claim 24, wherein said first analyzer is operable todetermine a magnitude of a dominant frequency above a thresholdfrequency, proximate individual pixels in said first hypotheticalimages; and said second analyzer is operable to determine a magnitude ofa dominant frequency above a threshold frequency, proximate individualpixels in said second hypothetical images.
 26. The detector of claim 25,wherein said first and second analyzers perform wavelet analyses. 27.The detector of claim 25, wherein said first and second analyzersperform Fourier analyses.
 28. The detector of claim 25, furthercomprising a difference calculator for calculating the differencebetween pixels in said immediately next field and said immediatelyprevious field.
 29. The detector of claim 17, wherein said processor isoperable to detect a 3:2 pull down sequence in said series of fields, inresponse to determining relatively high assessed vertical frequencycontent for two non-sequential ones of said first hypothetical imagesfor every five of said currently processed fields.
 30. The detector ofclaim 17, wherein said processor is operable to detect a 2:2 pull downsequence in said series of fields, in response to determining relativelyhigh assessed vertical frequency content for every alternating one ofsaid series of fields.
 31. A device for detecting a field sequence of aseries of fields of video, said device comprising: means for processingeach of said series of fields to construct at least a portion of a firsthypothetical image formed by interleaving alternating rows from thecurrently processed field and rows from an adjacent field; means forassessing the frequency content along the vertical direction of saidfirst hypothetical image to determine the presence of high frequencyvertical artifacts in said first hypothetical image; means for detectingsaid field sequence in said series of fields, by analyzing said assessedfrequency content for several of said series of fields.
 32. Computerreadable medium storing processor readable instructions, that whenloaded at a computing device operable to decode video into a pluralityof video fields, adapt said computing device to detect a field sequenceof said series of video fields in accordance with the method of claim 1.33. A video device comprising a detector for determining a fieldsequence, formed in accordance with claim
 25. 34. The method of claim 1,wherein said currently processed field and said adjacent field areformed from adjacent frames in a series of video frames, to detect framerepetition.
 35. A method comprising: for each field of a plurality ofvideo fields, constructing at least a portion of an associatedhypothetical image formed by interleaving rows from the field and rowsfrom an adjacent field in the plurality of video fields; assessing thefrequency content along the vertical direction of each of the associatedhypothetical images to determine the presence of high frequency verticalartifacts therein; determining a field sequence in the plurality offields from the sequence of those associated hypothetical imagesdetermined to have high frequency vertical artifacts.
 36. A method ofprocessing a series of fields of video, said method comprising: for eachof the fields, assessing frequency content along the vertical directionof an associated hypothetical image formed by interleaving alternatingrows from the field and rows from an adjacent field; classifying each ofthe associated hypothetical images as having or not having highfrequency vertical artifacts based on the assessing.
 37. The method ofclaim 36, further comprising determining a field sequence in the seriesof fields from the sequence of the associated hypothetical imagesclassified as having high frequency vertical artifacts.
 38. The methodof claim 36, further comprising forming frames of de-interlaced video,by combining fields and adjacent fields in the series of fields, foreach field for which its associated hypothetical image is classified asnot having high frequency vertical artifacts.
 39. The method of claim36, wherein the assessing comprises performing a frequency analysis ofsets of pixels within columns about addressed pixels of the hypotheticalimage.
 40. The method of claim 39, wherein the assessing comprisesperforming a wavelet analysis of the sets of pixels.
 41. The method ofclaim 39, wherein the assessing comprises performing a Fourier analysisof the sets of pixels.
 42. The method of claim 37, wherein thedetermining comprises recognizing a defined pattern of fields for whichthe associated hypothetical images are classified as having highfrequency vertical artifacts.
 43. The method of claim 36, wherein theassessing comprises performing multiple frequency analyses along columnsof each associated hypothetical image.
 44. The method of claim 36,wherein the assessing comprises determining magnitudes of dominantfrequency components in a vertical direction, above a thresholdfrequency, proximate addressed pixels in the associated hypotheticalimage.
 45. The method of claim 44, wherein the assessing comprisesanalysing a distribution of the magnitudes for multiple addressed pixelsand the classifying is performed based on the distribution.
 46. Themethod of claim 45, further comprising filtering the distribution of themagnitudes, using a distribution of magnitudes of dominant frequencies,above the threshold, in a hypothetical image not having verticalartifacts.
 47. The method of claim 37, wherein the field sequence isdetermined to be a 3:2 pull down sequence, in response to classifyingassociated hypothetical images for two non-sequential fields in everyfive fields of the series of fields, as having high frequency verticalartifacts.
 48. The method of claim 37, wherein the field sequence isdetected as a 2:2 pull down sequence, in response to classifyingassociated hypothetical images for every second field of the series offields, as having high frequency vertical artifacts.
 49. The method ofclaim 36, wherein each associated hypothetical image is formed byinterleaving alternating rows from a field and rows from its immediatelynext field, the method further comprising: for each of the fields,assessing frequency content along the vertical direction of a secondassociated hypothetical image formed by interleaving alternating rowsfrom the field and rows from its immediately previous field; classifyingeach of the second associated hypothetical images as having or nothaving high frequency vertical artifacts based on the assessing offrequency content along the vertical direction of the second associatedhypothetical image.
 50. The method of claim 49, wherein a field and itsimmediately previous field are de-interlaced in response to classifyingits second associated hypothetical image as not having high frequencyvertical artifacts.
 51. The method of claim 49, further comprisingforming a difference between pixels in the immediately previous fieldand pixels in the immediately next field, and wherein said assessingcomprises using the difference, for each of said fields to classify eachof the associated hypothetical images and the second associatedhypothetical images.
 52. The method of claim 37, wherein the fieldsequence is detected as a m:n:l:k pull-down sequence, in response toonly classifying, in m+n+l+k adjacent fields, the m^(th); m+n^(th);m+n+l^(th) and m+n+l+k^(th) associated hypothetical images as havinghigh frequency vertical artifacts.